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OBJECTIVE: 

fn  Sfcentury  Navy,  war  fighting  objectives  include  technological  superiority,  reduced  staffing,  and  an 
affordable  force.  Given  the  requirement  of  superior  performance  with  fewer  personnel,  the  Nay  has  a  cntical 
for  research  that  addresses  human  performance  enhancement.  The  present  research  investigated  how  cognitively  based 
design  principles  enhance  human  performance  in  complex  technological  interactive  task  environments.  Becau 
successful  this  research  requires  knowledge  of  multiple  disciplines,  a  multidisciplinary  research  team  was  required. 
Our  collaborative  team  included  researchers  trained  in  computational  engineering,  computer  science,  cognitive 
science  and  cognitive  psychology. 

^  specific  goal  of  this  project  was  to  investigate  cognitively  based  virtual  environments  (VE)  and  interfaces  that 
optimize  meteorological  and  oceanographic  (METOC)  analyst  training  and  performance  during  mine  counter 
measure  (MCM)  operational  tasks.  METOC  personnel  depend  on  advanced  visualization  tools  to  obtain  information 
that  wifi  aid  their  decision  making  in  tactical  planning  tasks.  These  visualization  systems  render  copious  amounts 
of  data  in  order  to  represent  complex  scientific  phenomena.  In  the  present  context,  the  data  show  historical  or 
forecasted  changes  in  oceanographic  conditions  over  time.  In  a  virtual  METOC  visualization  environment,  the 
analyst  was  “immersed”  in  the  simulation,  which  was  hypothesized  to  allow  more  insight  into  the  dynamic 
processes  of  coastal  regions  coupled  with  the  operational  task  at  hand. 

Analysts  can  interact  with  a  VE  via  a  virtual  interface  (VI)  in  order  to  build  an  understanding  of  represented 
oceanographic  conditions.  In  other  words,  the  analyst  uses  the  virtual  interface  to  focus  on  infomiation  used  to  bur d 
a  mentaf  model  of  the  simulated  environment,  or  a  “situation  model”  (Durso  &  Gronlund,  ^L^tuationmod 
is  a  term  used  to  describe  a  person’s  understanding  of  a  represented  environment  (e.g.,  Doane  &  Sohn,  2000, 

Doane  Sohn  McNamara,  &  Adams,  2000;  Sarter  &  Woods,  1991;  Sohn  &  Doane  2000).  In  the  present  context, 
the  analyst  uses  the  situation  model  of  the  oceanographic  conditions  to  inform  his  or  her  tactical  planning  decision¬ 
making  (e  g  determine  if  cloud  cover  allows  a  stealth  approach  to  shore;  rate  the  probability  of  a  sonar  contact 
Sg  fmSbaseZ  oceanographic  conditions  impacting  sonar  image  qtralityt  see  Trafton  et  al  2000).  The  goal 
of  *e  application  designer,  then,  is  to  provide  analysts  with  interface  tools  that  facilitate  the  contraction  of  a 
situatioSmodel  that  will  support  the  highest  quality  tactical  planning  decision  making.  In  other  words,  the  virtual 
interface  should  enhance  human  analyst  performance. 

The  snecific  focus  of  the  project  was  on  the  scientific  visualization  of  oceanographic  data  used  in  the  study  of 
sedimentation  transport.  The  study  of  sedimentation  transport  is  important  for  the  Navy  MCM  community,  in 
instances  where  historical  records  of  past  mine  contacts  must  be  examined.  For  example,  MCM  personnel  may 
need  to  determine  if  sediment  has  concealed  previously  detected  mines  since  the  last  time  they  were  encountered. 
The  objective  of  this  project  was  to  use  principles  of  human-computer  interaction  and  applied  cognitive  science  to 
address  human  information-processing  and  system  design  issues  relevant  for  scientific  visualization  tasks,  sue  as 
the  study  of  sedimentation  transport.  This  objective  included  several  subgoals.  One  set  of  subgoals  involved  the 
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design  of  the  visualizations  and  the  interface  for  that  visualization.  Another  set  of  subgoals  involved  the  nature  of 
the  virtual  environment  used  to  display  the  scientific  visualizations. 

Many  ofthe  problems  we  face  today  involve  complex  relationships  between  variables  representing  interrelated 
processes  It  is  computationally  expensive  both  to  model  these  processes  and  to  analyze  and  understand  the  huge 
amounts  of  data  generated.  Often  the  best  way  to  analyze  these  relationships  is  visually,  with  some  form  of  graphic 
representation.  In  the  past,  it  was  sufficient  to  display  any  representation  that  the  computer  could  render,  forcing 
much  information  as  possible  into  one  image.  We  have  reached  a  computational  threshold  where  we  can  often 
display  more  information  than  is  useful,  and  so  now  we  are  concerned  with  the  perceptual  issues  of  combining 
colors! primitives  (i.e.,  graphical  building  blocks),  and  textures  such  that  we  can  gain  the  greatest  understanding  of 

what  is  rendered. 

Producing  meaningful  visualizations  now  means  maximizing  how  well  the  information  they  convey  is  perceived. 
What  types  of  visual  techniques  will  best  show  information  about  complex  relationships  between 'variables, 
especially  when  used  in  combination  with  other  techniques?  Many  times  there  is  no  single  solution  but  hybrid 
solutions^  such  as  using  one  technique  to  “browse”  data  on  a  macro  scale  to  identify  areas  of  interest  then  using 
another  method  to  show  detailed  information  within  a  smaller  context  (“drill  down”),  often  provide  the  best  answer. 
Within  this  context,  we  have  developed  a  new  method  for  conveying  information  about  size-fractionated 
sedimentation  concentrations  (browsing)  within  a  larger  visualization  of  related  variables.  Aerate  i nod ehngof 
environmental  processes  such  as  sedimentation  (i.e.  the  settling  of  clay,  silt  and  sand  to  the  bottom  of  a  body  of 
water)  and  aerosol  distributions  (e.g.,  precipitation,  dust,  pollutants,  or  biochemical  toxins)  is  better  achieved  1 
concentrations  are  more  accurately  differentiated  by  particle  size.  Computing  and  rendering  information  related  to 
multiple  scalar  fields,  where  before  there  was  only  a  single  field,  is  a  significant  challenge  for  the  area  of 
visualization  design. 

One  goal  of  our  visualization  design  has  been  to  enable  analysts  at  Stennis  NRL  to  study  complex  littoral  region 
nrocesses  such  as  how  a  Navy  diver’s  visibility  might  be  affected  by  changes  m  weather  and  the  surrounding 
underwater  environment,  including  sedimentation  (Keen,  Vickery,  Flynn,  Stavn,  &  McBride  200  1 ,  Vickeryetal., 
2002a).  Additionally,  Navy  MCM  operations  can  use  the  information  on  erosion  and  deposition  of  sediment  to 
understand  burial  characteristics  of  mines  in  specific  underwater  regions. 


In  addition  to  the  concerns  about  designing  a  visual  representation  ofthe  data,  analysts  must  be  able  to  lntera 
the  representation  effectively  once  it  is  completed.  In  order  to  design  a  usable  interface  for  such  a  visualizatio  , 
analyst’s  information  requirements  must  be  understood  and  incorporated  into  the  interface.  The  user  with  whom  we 
were  most  concerned  was  an  expert  in  sedimentation  transport,  Dr.  Tim  Keen.  Dr.  Keen  has  developed  a 
computational  model  of  sedimentation  transport  that  can  calculate  the  characteristics  ofthe  ocean  bottom  given  a  set 

of  remotely  sensed  data  as  input. 

Dr  Keen’s  tasks  with  his  model  are  two-fold.  First,  he  must  diagnose  problems  in  the  data  sets  that  his  model 
generates  based  on  his  knowledge  of  oceanographic  processes.  Problems  that  he  finds  may  be  caused  by  errors  m 
his  model’s  calculations  or  by  a  bad  set  of  input  values.  Using  traditional  methods  of  analysis,  finding  and 
diagnosing  such  problems  in  a  data  set  is  a  time-consuming  process. 

The  second  task  Dr.  Keen  must  accomplish  with  his  model  is  understanding  the  state  of  the  ocean  from  the  model  s 
output.  That  is,  he  must  integrate  the  various  pieces  of  data  mentally  in  order  to  make  predictions  about  the  fii 

states  of  the  ocean. 

Cognitive  task  analysis  is  one  method  that  has  been  used  to  understand  user  information  requirements  for  the  design 
of  interfaces  (e.g.,  Endsley,  1993;  Schraagen,  Chipman,  &  Shalin,  2000).  In  our  cognitive  task  analysis,  we 
focused  on  gaining  an  understanding  of  how  our  SME,  Dr.  Keen,  uses  the  information  he  has  about  an 
oceanographic  environment  to  diagnose  problems  with  his  model  of  sedimentation  transport.  Our  first  goal  m  the 
analysis  was  to  determine  the  relationship  between  different  pieces  of  information,  according  to  our  SME.  Knowing 
the  relationships  between  aspects  ofthe  data  gave  us  a  better  understanding  of  when  certain  information  would  be 
important  for  Dr.  Keen  to  be  able  to  access  quickly.  Our  second  goal  was  to  develop  a  process  diagram  that 
described  the  steps  Dr.  Keen  took  to  diagnose  problems  with  his  model.  The  process  diagram  would  also  help  us 
design  an  interface  that  could  present  useful  information  to  Dr.  Keen  at  the  right  times  and  in  the  right  way. 
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Immersive  Versus  Nonimmersive  Environments 

With  the  increase  in  VE  technology,  some  researchers  have  hypothesized  that  a 

VE  would  lead  to  better  understanding  of  information  presented  m  the  VE  (e.g  Slater  &  Wilbur,  1997).  Presence 
can  be  thought  of  as  the  sense  that  the  virtual  world  is  a  real  environment  that  the  user  is  actually  standing  in  and 
interacting  with.  When  an  environment  is  thought  to  offer  a  high  sense  of  presence,  it  is  referred  to  as  an 
“immersive”  environment.  The  impact  of  immersion  or  presence  on  a  user’s  understanding  of  information  is  also 
important  to  understand.  Immersive  systems  are  often  very  expensive  to  purchase  and  to  maintain.  If  a  less 
immersive  environment  can  support  the  information-processing  requirements  needed  for  the  scientific  visualization 
of  oceanographic  data  as  well  as  an  immersive  environment,  then  the  Navy  should  consider  using  the  less  expensive 
alternative.  The  impact  of  immersion  on  user  understanding  of  information  was  examined  in  another  experiment 

conducted  for  this  project. 

Svatial  Cognition  in  Virtual  Environments  . 

There  are  many  differences  between  the  real  world  and  even  the  most  hi#-fidelity  virtual  environment  Some  of  the 
most  documented  differences  involve  spatial  cognition  (e.g..  Cutting,  1997;  Loomis,  Blaschovich  &  Beall,  1999). 
Because  visualization  has  an  inherent  spatial  component,  it  is  important  to  understand  how  spatial  cognition  in  a 
VE  differs  from  the  real  world.  Such  differences  may  be  important  for  the  design  of  appropriate  visualizations  and 

interfaces  in  VEs. 

Numerous  theories  have  been  developed  about  how  people  keep  track  of  the  elements  of  their  environments  in  the 
real  world.  Two  of  the  major  theories  in  the  literature  are  the  configural  theory  and  the  egocentric  theory  of  spatial 
cognition.  The  configural  theory  states  that  people  keep  track  of  objects  in  space  around  them  by  storing 
information  about  the  object-to-object  relationships  they  encounter  (e.g.,  Sholl  &  Nolm,  1997).  Such  information 
leads  to  the  formation  of  “cognitive  maps,”  that  can  be  considered  mental  representations  that  have  the * 
models  of  an  environment.  Cognitive  maps  are  believed  to  be  allocentnc,  or  independent  of  any  specific  onentation 
taken  by  the  person  in  the  environment.  The  egocentric  theory  states  that  people  keep  track  of  the  relationships 
between  objects  in  an  environment  and  their  own  body’s  position  m  the  environment  (e.g.,  Wang  &  Spelke,  2000). 
The  egocentric  theory  suggests  that  object  locations  are  represented  in  transient  “snapshots.  These  snapshots  can 
be  recalled  when  a  person  is  not  in  an  environment,  but  they  can  only  be  recalled  from  the  particular  orientation  m 

which  they  were  “taken.” 

Given  these  two  theories  and  the  previous  research  on  spatial  cognition  performed  in  the  real  world,  we  conducted  a 
set  of  studies  to  determine  what  representations  were  used  when  performing  spatial  tasks  in  a  VE.  Our  investigation 
led  to  a  study  of  the  strategies  people  can  use  in  order  to  effectively  navigate  in  VEs,  and  the  results  from  that  study 
may  guide  future  work  on  designing  navigation  tools  for  VE  interfaces. 

Taken  together  the  results  of  our  efforts  produced  valuable  information  about  the  design  of  scientific  visualizations 
and  interfaces  for  the  visualizations.  Although  our  objectives  focused  on  scientific  visualization  of  oceanographic 
data,  much  of  our  work  should  generalize  to  other  areas  of  scientific  visualization. 


APPROACH: 

For  this  research  program,  our  goals  included  designing  visualizations  designing  the  interface  for  the  visualization, 
studying  the  nature  of  spatial  cognition  in  the  virtual  environment  used  to  display  the  scientific  visualizations,  as 
welfas  studying  the  impact  of  immersion  on  understanding  scientific  data  presented  in  a  VE.  Our  applied  emphasis 
included  visits  to  Pensacola  MCM  facilities  to  talk  with  METOC  analysts  (e.g.,  LTjg  Jay  Jones  at 
NAVTRAMETOCFAC),  and  visits  to  NAVO  and  NRL  at  Stennis  to  talk  with  METOC  analysts  there  (e.g.,  Dr. 
Timothv  Keen  and  Steve  Lingsch  at  NAVO  Stennis).  The  goal  of  these  discussions  was  to  determine  the  content 
and  nature  of  the  information  these  analysts  use  during  their  decision-making  process.  The  results  from  these 
discussions  and  formal  task  analyses,  coupled  with  our  theoretical  findings,  enabled  us  to  develop  cognitively  based 
design  principles  for  MCM  METOC  VEs  and  Vis.  The  specific  details  of  the  approach  we  used  for  each  of  our 
goals  will  be  discussed  in  turn. 

Th^approach  for  the  visualization  portion  of  this  project  was  to  develop  new  visualization  algorithms  for  complex 
physical  problems  and  to  compare  them  with  other  state-of-the-art  techniques  m  terms  of  rendering  speed  and  user- 
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nprreived  effectiveness.  User  studies  comparing  the  new  visualizations  to  the  state-of-the-art  techniques  were 
developed  with  a  focus  on  the  interactivity  of  the  visualizations,  as  well  as  on  the  effectiveness  of  the  visualization 
as  a  means  of  presenting  information  to  the  users.  Further  information  on  the  visualization  can  be  found  online  at 
htip-//WWW  F.Ttf'.MsStaie.F.du/~rvickery/. 

For  the  designof  the  interface,  a  cognitive  task  analysis  was  performed  to  help  understand  the  information 
requirements  and  procedures  for  studying  sedimentation  transport  m  the  littoral  ocean  environment.  The  cognitive 
ask  analysis  began  with  several  interviews  with  our  SME.  Following  the  initial  interviews,  we  used  several 
echniques  to  gather  more  information  about  how  the  SME  used  his  model  to  analyze  sedimentation  transport  data. 
The  first  technique  was  a  hierarchical  card-sorting  task.  In  this  task,  the  SME  was  given  many  index  cards,  each 
having  the  nan^ of  one  variable  written  on  it.  The  SME  was  asked  to  group  the  index  cards  by  grates 
and  tcf  label  the  resulting  groups.  The  SME  then  combined  the  groups  based  on  greatest  similarity  and  labeled  the 
netjarger  gmups This  process  continued  until  all  the  index  cards  were  in  one  large  group  labeled  variables  ’ 

The  result  of  this  grouping  was  a  hierarchical  sorting  of  the  variables  that  could  be  used  to  represent  the  SME  s 
representation  of  the  sedimentation  transport  domain  (Doane,  Pellegrino,  &  Klatzky,  1990).  Figure  1  shows  the 
knowledge  representation  diagram  that  resulted  from  this  task. 

After  examining  the  hierarchy  of  variables,  we  watched  the  SME  perform  his  diagnostic  tasks.  The  notes  taken 
from  that  process  were  used  to  build  an  initial  process  diagram  showing  the  steps  the  SME  used  to  examine  the 
oceanographic  environment.  This  diagram  was  refined  through  interviews  and  more  observations  of  the  SME 
performing  his  tasks.  The  purpose  of  this  process  diagram  was  to  inform  our  interface  design  by  describing  when 
and  how  information  was  used  during  the  task. 

Figure  2  shows  the  process  diagram  for  the  portions  of  the  SME  tasks  on  which  we  focused.  Specifically,  it  shows 
the  process  of  searching  for  and  diagnosing  anomalous  points  in  an  oceanographic  data  set.  Based  on  the  cogmtiv 
Jask  analysis  we  designed  storyboards  for  an  interface  and  refined  our  ideas  by  using  rapid  prototyping  tools  and 
further  dfscussions  with  the  SME.  The  interface  was  later  tested,  and  the  results  of  the  tests  are  discussed  m  the 
“Immersive  Versus  Nonimmersive  Environments”  sections  of  this  report. 

Immersion  Versus  Nonimmersive  Environments  A  ^ 

Once  the  interface  design  was  finalized,  it  was  implemented  in  two  systems.  The  systems  used  were  a  flat-screen 
computer  and  a  four-walled  Cavernous  Automatic  Virtual  Environment  (CAVE).  Figure  3  shows  the  flat-screen 
version  of  the  interface  and  the  visualization,  whereas  Figure  4  shows  the  CAVE  version  of  the  interface  and  the 
visualization.  These  systems  were  chosen  because  of  the  relative  amount  of  immersion  inherent t  in  their  use. 
is  the  flat-screen  computer  is  not  generally  considered  as  immersive  as  a  CAVE  (Cruz-Neira  1998).  By 
implementing  the  interface  in  two  systems  that  differed  in  levels  of  immersion,  we  could  test  the  impact  of 
immersion  on  the  scientist  comprehension  of  the  visualization. 

When  the  interfaces  were  completely  implemented,  our  SME  served  as  a  subject  in  an  experiment  investigating  the 
impact  of  level  of  immersion  on  scientific  data  comprehension.  Before  the .experimental  tasks  were  given  to  the 
SME  he  was  trained  to  use  both  interfaces.  Training  consisted  of  having  him  perform  simple  tasks  that  required 
the  use  of  each  part  of  the  interface  in  turn.  For  example,  he  was  first  asked  to  navigate  to  a  specific  point  m  the 
ocean  environment.  This  meant  that  he  had  to  use  the  navigation  function  of  the  interface,  such  as  manipulating  t 
wand  in  the  CAVE  or  using  the  mouse  to  press  the  movement  icons  in  the  flat-screen  interface.  Once  the  SME  was 
trained  on  both  interfaces,  the  experimental  tasks  began. 

The  tasks  used  in  the  experiment  were  designed  to  test  the  SME’s  comprehension  of  the  data  being  visualized.  For 
each  task,  a  certain  point  in  an  ocean  environment  was  manipulated  to  cause  an  anomaly  m  the  data.  For  example, 
the  SME  might  be  presented  with  an  environment  in  which  the  ocean  bottom  was  composed  of  fine  silt  The 
anomalous  point  might  have  a  very  coarse  mud  in  place  of  the  silt,  changing  the  properties  of  the  currents,  sediment 
entrainment  etc.  The  SME  would  first  have  to  find  the  anomalous  point  in  the  environment,  then  diagnose  wha 
caused  the  anomaly.  In  the  previous  example,  that  would  involve  determining  that  the  sediment  coarseness  had 
been  manipulated  for  the  anomalous  point.  If  the  SME  felt  he  could  not  make  progress  on  any  part  of  a  task,  he 
could  request  hints  from  the  experimenter.  While  he  used  each  interface,  we  recorded  the  amount  of  time  it  took  our 
SME  to  use  commands,  find  anomalous  points,  and  diagnose  problems.  These  reaction  times  were  compared  across 
the  systems,  as  were  the  number  of  hints  required  to  solve  the  problems  accurately. 
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Figure  1.  Representation  of  SME’s  conceptual  structure  of  the  variables  involved  in  sedimentation  transport. 


Svatial  Cognition  in  Virtual  Environments 

Real  world  studies  of  spatial  cognition  have  focused  on  understanding  the  underlying  mental  representations  used 
SSSwUle  igatog  teough  space  (e.g,  Shelton  &  McNamara,  1997;  Shell  &  Bartda.  2002).  In  order 
to  examine  the  ways  that  people  navigate  in  VEs,  our  research  adapted  a  research  paradigm  used  for  studying  spatial 
cognition  in  the  real  world  (see  Wang  &  Spelke,  2000).  The  paradigm  we  used  contrasted  the  configural  and 
egocentric  theories  of  spatial  cognition.  These  theories  predict  different  effects  of  being  disoriented  m  an 
environment.  According  to  the  configural  theory,  a  person  represents  the  configuration  of  an  environment  throug 
object-to-object  relationships  instead  of  through  self-to-object  relationships.  Thus,  the  configural  theory  suggest 
that  people  should  still  be  able  to  access  the  relative  positions  of  objects  in  an  environment,  even  after  losing  their 

sense  of  orientation. 

Bv  contrast  according  to  the  egocentric  theory,  a  person  must  access  self-to-object  relationships  in  order  to  recall 
the  positions  of  objects  in  space.  Thus,  the  egocentric  theory  suggests  that  people  should  lose  their  ability  to  access 
the  relative  positions  of  objects  following  disorientation.  If  this  theory  is  correct,  then  when  information  about  self¬ 
position  is  lost,  accessing  self-to-object  relations  should  become  difficult. 
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Figure  2.  Process  diagram  for  SME’s  analysis  of  anomalous  points  in  oceanographic  data. 

In  order  to  determine  which  type  of  representation  people  are  using  in  the  VE,  we  had  subjects  learn  the  locations  of 
six  obiects  in  a  virtual  world.  Figure  5  shows  an  example  of  a  subject  studying  the  locations  of  objects  m  the 
CAVE  Once  the  subjects  had  learned  the  locations,  they  were  asked  to  indicate  the  object  locations  when  the 
obiects  were  occluded  from  view.  This  was  first  done  while  the  subjects  were  oriented  in  the  environment  and  later 
while  thev  were  disoriented  wearing  a  blindfold.  Figure  6  shows  an  example  of  a  blindfolded  subject  pointing  to 
objects  using  the  wand.  From  subject  responses,  a  configuration  error  was  calculated.  Configuration  error  was 
thoueht  to  describe  the  tendency  for  a  subject  to  recall  the  object-to-object  relationships  accurately.  The  difference 
in  configuration  error  between  oriented  and  disoriented  phases  indicated  the  type  of  spatial  representation  the  subjec 
was  using  If  the  subjects  showed  an  increase  in  configuration  error  following  disorientation,  this  indicated  the 
Sect  was  using  an  egocentric  representation.  If  the  subject  showed  no  increase  m  configuration  error  this 
indicated  the  suW  ect  was  using  a  configural  representation.  Findings  from  our  initial  study  showed  that  there  are 
systematic  individual  differences  in  the  types  of  representations  used  by  individuals  when  they  navigate  m  a  VE. 
Further  Ihe  subjects  were  shown  to  use  strategies  to  improve  their  performance  on  spatial  tasks.  Our  investigation 
of  spatial  cognition  in  VEs  later  expanded  into  a  study  of  the  types  of  strategies  subjects  could  use  to  perform 
spatial  tasks fn  a  VE.  We  specifically  examined  how  strategies  used  by  subjects  were  affected  by  the  type  of 

representation  they  were  using. 
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Figure  4.  Screenshot  of  the  immersive  CAVE  version  of  the  interface  and  visualization. 


BEST  AVAILABLE  COPY 


Figure  5.  Example  of  a  subject  studying  the  locations  of  objects  in  the  spatial  cognition  experiment. 


Figure  6.  Example  of  a  blindfolded  subject  indicating  the  locations  of  objects  in  the  spatial  cognition  experiment. 
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A  CCOMPLISHMENTS  (throughout  award  period): 


We  Srtfdtv  M  a  basic  system  to  visualize  the  aval table  scalar  and  vector  variables  produced  by  the 
oceanographic* sedimentetion modeb  ’ 

rai'dation  mcflioddBi  allowsmultiple  related  scalar  variables  to  be  viewed  with  other  relevant  data  (Vickery  et 
i  Thk  method  suffered  from  two  major  shortcomings:  Sedimentation  particles  of  vastly  different  sizes 

SLSas The  »me Sind  fc“wSe  too  many  colom  to  distinguish  distinctly  between  the  different  scalar 
fiel*  Therefore  the  algorithm'was  enhanced  to  view  multiple  related  scalars  more  effectively  by  mcorporatmg  a 
better  ranitc  of  colors  and  sizes  (Vickery  et  .1.,  2002b).  In  the  enhanced  version,  colons  used  to  separate  up  to  20 
related  scalars  or  "bins"  (in  this  case  differentiated  by  gram  size)  into  4  general  categones  (representing  clay,  silt, 
sanT Ind  gravel)  Within  the  categories,  varying  size  glyphs  (i.e,  visualization  constructs  used  to  represent  scalar 
valuesTshow  the  magnitudes  for  each  bin.  Smaller  glyphs  show  magnitudes  for  smaller  grain  size  bins  and  larger 

elvnhs  for  larger  grain  size  bins.  This  combination  of  color  and  size  provides  two  levels  of  detail  to  help  the 

fSt  deteSe  fte  types  of  sediment  present  in  the  water.  The  addition  of  logarithmic  scaling  to  the  computation 
for  the  number  of  glyphs  and  how  they  are  allocated  to  different  regions  within  the  VE  was  another  major 
enhancement. 

To  determine  if  our  implemented  visualization  method  was  effective  for  viewing  large  datasets,  we  chose  two 
criteria-  interactivity  in  the  VE  and  effectiveness  as  an  information  visualization  method.  Because  our 
ImpfomtSaSn  readers  the  infomtatiot,  as  pariiclcs,  we  t tompared  foe  1 perfotmaucc  of  I  w»h 

two  of  the  most  common  methods  for  displaying  particles  m  the  VE  (yickery  et  al  2003a)  This  study helped us 
determine  how  parameters  needed  to  be  set  to  maximize  the  interactivity  of  our  implementation  We  were  able  to 
sho“  ^method  was  at  least  as  interactive  as  the  other  two  in  the  worst  case,  and  up  to  44  times  more 

interactive  in  the  best  case. 

We  have  just  completed  a  pilot  study  on  the  effectiveness  of  our  method  as  an  information  visualization  technique 
for  browsing  largePdatasetSP  The  information  obtained  is  being  used  to  design  a  formal  user  study  that  will  focus  on 
how  well  the  related  scalar  values  are  represented  as  texture  mapped  glyphs,  and  also  how  well  the  method  compares 
with  another  state-of-the-art  visualization  methods  (Vickery  et  al.,  2003b). 

We  have  also  enhanced  an  existing  library  of  VE  interface  programming  tools,  called  widgets,  that  can  be  used  to 
prototype  both  a  developer's  graphical  interface,  as  well  as  a  custom  interface  to  support  specialized  user  studies 
(Brou  &  Doane  2003).  One  particularly  useful  widget  was  created  as  an  aid  to  help  ana  ysts  understand  the  effect 
of  logarithmic  scaling  on  the  visualization  of  scalar  variables  with  a  range  of  values  too  large  to  represent  dearly. 
This  interactive  widget  maps  scalar  values  to  a  color-mapped  legend  such  that  foe  analyst can  see  the  exact  value  of 
the  scalar  field  represented  by  a  specific  color  on  the  logarithmic  scale  (Noble  et  al.,  2003). 

fo^rpast^sedimentation  transport  analysis  involved  examining  printouts  of  2-D  slices  of  the  ocean  man  area. 
These  slices  showed  numerical  values  for  one  variable  at  regular  intervals  m  an  ocean  region.  Because  the  analysis 
of  sedimentation  transport  required  a  mental  model  of  the  status  of  many  variables  m  an  area  and  how  the  variables 
interacted,  the  traditional  method  of  analysis  was  an  extremely  demanding  task. 

In  our  approach,  we  first  completed  a  cognitive  task  analysis  for  an  oceanographic  sedimentation  transport  expert 
From  thkf analysis,  we  generated  a  knowledge  representation  diagram  and  a  process  diagram  on  which  an  interface 
design  was  based.  We  predicted  that  the  time  required  for  sedimentation  transport  analysis  of  a  littoral  region 
would  significantly  decreased  by  using  a  VE  with  the  interface  designed  from  our  task  analysis,  because  the  VE 
and  interface  would  facilitate  the  effective  building  of  a  qualitative  mental  model  of  the  littoral  region  (Trafton  e 

al.,  2000). 

Our  interface  design  was  implemented  in  both  a  nonimmersive  and  an  immersive  environment  Our  SME  described 
our  interface  as  “an  order  of  magnitude  better”  than  the  traditionally  used  system  for  sedimentation  transport  data 
analysis  Because  the  interface  was  implemented  in  multiple  types  of  systems,  we  were  able  to  use  it  in  a  study  of 
the  impact  of  immersion  on  the  comprehension  of  scientific  visualizations.  The  details  of  that  study  are  reported  m 
the  “Immersive  Versus  Nonimmersive  Environments”  sections  of  this  report. 
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Tte  results' ofthe  Experiment  examining  the  impact  of  immersion  on  comprehension  of  scientific  visualizations 
showed  that  immersion  did  not  enhance  comprehension  of  scientific  visualizations,  as  the  SME  was  able  to 

i  tp  aii  tacks  in  both  the  immersive  and  the  nonimmersive  environments.  Further,  the  SME  took  longer  to 
“ZlEte  the  tasks  and  required  more  hints  about  the  tasks  when  he  used  the  immersive  environment  to  study  the 
visualizations  Figure  7  shows  the  mean  amount  of  time  needed  by  the  SME  to  locate  anomalous  points  in  the 
immersive  and  nonimmersive  environments.  Figure  8  shows  the  mean  number  of  hints  needed  for  solving  tasks  in 
the  two  environments.  The  SME  reported  that  both  the  immersive  and  the  nonimmersive  environments  were 
Superior  to  die  traditional  methods  of  data  analysis.  The  results  of  this  study  are  currently  being  prepared  for 

publication. 
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Figure  7.  Mean  time  required  (in  seconds)  for  the  SME  to  locate  anomalous  points  in  the  oceanographic  data  using 
the  immersive  and  nonimmersive  environments. 


wf»LpSSo spatial  cognition  during  this  project.  He  fct  experimert  showed 
that  there  are  individual  differences  in  the  representations  that  people  use  to  navigate  in  VEs.  That  is,  about  1/3  of 
thE  subjects  we  tested  formed  configural  representations  ofthe  VE,  whereas  2  3  formed  egocentric  representations. 
Further  the  subjects  who  formed  egocentric  representations  were  able  to  employ  a  strategy  of  assumrng  a  familiar 
oricmioi  (K,  Samng  m  egoceLc  snapshot  of  a  previously  viewed  orientation)  in  order  to  perform  fte  object 
localization  task  effectively  following  disorientation.  Figure  9  shows  an  example  of  a  subject  who  is  initially 
facing  the  plant  in  our  experimental  environment  before  being  disoriented.  Following  disorientation,  ^subject 
might  assume  a  familiar  or  an  unfamiliar  orientation.  These  two  possibilities  are  illustrated  on  the  right  side  o 
Figure  9.  On  the  top  right,  the  subject  points  as  if  facing  a  familiar  orientation  (i.e.,  towards  the  plant),  on  the 
bottom  right  the  subject  points  as  if  facing  an  unfamiliar  orientation  (re.,  towards  the  television).  Subjects  using 
an  egocentric  representation  chose  orientations  that  were  more  familiar  than  subjects  who  used  a  configural 
representation.  This  familiar  orientation  strategy  facilitated  their  recall  of  object  locations,  even  though  their  sense 
of  orientation  in  the  environment  had  been  lost.  Figure  10  shows  the  mean  distance  from  a  familiar  orientation 
chosen  by  configural  and  egocentric  subjects.  Our  second  experiment  showed  that  subjects  who  initially  used  a 
familiar  orientation  strategy  could  not  perform  the  task  effectively  if  they  were  not  allowed  to  use  that  strategy.  The 
experiments  on  spatial  cognition  have  led  to  three  conference  papers  and  one  paper  under  review  for  publication  m 
Spatial  Cognition  and  Computation. 
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Mean  Number  of  Hints  Needed 
to  Find  Inconsistent  Points 


Fieure  8  Mean  number  of  hints  needed  for  the  SME  to  complete  tasks  accurately  in  the  immersive  and 
nonimmersive  environments.  In  each  case,  the  SME  required  hints  about  the  location  of  anomalous  points  in  the 

oceanographic  visualization. 
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Figure  9.  Diagram  of  a  subject  selecting  a  high  or  low  distance  from  a  familiar  orientation.  On  the  left,  a  subject 
may  initially  have  a  heading  in  which  the  plant  is  directly  in  front  of  him  or  her.  If,  after  disorientation,  the  subjec 
assumes  the  plant  is  still  in  front  of  him  or  her  (i.e.,  top  right  figure),  that  subject  is  described  as  having  a  ow 
distance  from  a  familiar  orientation.  If,  after  disorientation,  the  subject  assumes  a  novel  orientation,  in  which  the 
5SSSS  front  of  him  or  her  (i.e.,  bottom  right  figure),  that  subject  is  described  as  having  a  high  distance  from 

a  familiar  orientation. 
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Figure  10.  Mean  distance  from  a  familiar  orientation  selected  by  subjects  as  a  function  of  mental  representation  m 
the  spatial  cognition  experiment. 


CONCLUSIONS: 


Presses  I" , .  ■ VE is  a  difficult  problem  with  .0  single  solution.  E^cdve 
visualTzations8for  VEs  are  designed  by  understanding  the  context  of  the  problem  and  the  relationships  between 
variables  to  determine  the  information  desired.  The  best  visualizations  are  often  combinations  of  existing  methods 
that  display  the  information  based  on  the  amount  of  detail  required.  Hence  some  techniques  work  best  to  browse 
data  over  a  large  area.  These  techniques  can  be  used  to  identify  specific  grid  locations  where  higher  deta 
techniques  can  be  used  to  obtain  more  specific  information. 

In  this  project,  we  discovered  that  the  texture  mapping  algorithm  was  most  effective  for  browsmg  large  areas  to 
understand  global  changes  in  sediment  concentration  at  a  glance.  More  detailed  techniques  were  used  to  show 
breakdown  of  related  scalar  variables  for  specific  grid  locations.  It  is  this  combination  of  techniques  used  in  the 
proper  context  that  has  provided  the  most  complete  information  for  the  analyst. 

Usfnfafo^ifivTfreoty-based  approach  to  the  design  of  the  virtual  interface,  we  successfully  built  a  tool  that 
facilitates  ascientist’s  ability  to  perform  oceanographic  data  analysis.  Our  interface  was  designed  to  lessen  the 
menS  workload  of  the  use?  giving  memory  support  tools  and  shortcuts  based  on  the  information  requirements  of 
the  scientific  visualization  tasks.  Our  SME  found  the  use  of  our  interface  to  be  superior  to  the  traditional  methods 
of  oceanographic  data  analysis. 


Immersive  Versus  Monimmersive  Environments  .  .  j  j 

In  our  experiment  on  the  impact  of  immersion  on  the  comprehension  of  scientific  visualizations  we  recorded 
reaction  times  to  complete  tasks  in  an  immersive  and  a  nonimmersive  version  of  our  interface  An  analysis  of  those 
reaction  times  showed  that  the  SME  required  significantly  more  time  and  more  hints  to 

immersive  CAVE  environment.  Both  of  these  findings  can  be  explained  by  the  fact  that  the  SME  got  lost  in  the 
visualization  in  the  immersive  condition.  That  is,  with  so  many  data  points  m  view  at  one  time,  the  SME  had 
difficulty  spotting  the  areas  of  interest  in  the  visualization  (see  Figure  7).  This  finding  is  important  for  at  least  two 
reasons  First  we  failed  to  support  the  idea  that  immersion  leads  to  better  comprehension  of  scientific  data. 

Second,'  the  findings  suggest  that  more  research  is  required  to  understand  when  an  immersive  VE  can  be  superior  to 
a  nonimmersive  environment. 
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Therelute S resS infpSTSon  have  shown  there  are  individual  differences  in  the  way  people 
1  h  Lt  nhiprK  in  snace  as  thev  navigate  in  a  VE.  That  is,  about  one-third  of  our  subjects  appeared  to  use  a 
configural^representation  whereas  two  thirds  of  our  subjects  showed  evidence  of  using  an  egocentric  representation, 
further  ^hep^ple  using  an  egocentric  representation  were  able  to  use  a  specific  strategy  to  complete  our  spatial 
tas^following  dEiorientation.  This  strategy  consisted  of  assuming  a  familiar  orientation  m  the  environment  and 
recallingan  orientation-specific  snapshot  in  order  to  respond  to  object  location  queries^  Future  research  m  interface 
de^gnforVE  navigation  might  be  able  to  take  advantage  of  our  findings.  Because  subjects  can  generate  appropriate 
strategies  that  optimize  the  use  of  their  mental  representations,  an  interface  design  might  incorporate  elements  that 
facilitate  the  use  of  such  strategies. 


SIGNIFICANCE: 

Acciuate^nodriing^of  environmental  processes  such  as  sedimentation  (i.e.,  clay,  silt  and  sand)  and  aerosol 
distributions  (e.gnf precipitation,  dust,  pollutants,  or  biochemical  toxins)  is  better  achieved  if  the  concentrations  are 
more  accurately  differentiated  by  particle  size.  Computing  and  rendenng  information  relating  to  multiple  sea  ar 
fields  rather  than  just  a  single  field  as  before  is  a  significant  challenge  for  this  area,  especially  when  the  desired 
result  is  to  understand  related  variables  underlying  complex  physical  processes. 

The  benefits  of  our  research  included  an  enhanced  knowledge  of  emerging  computer  technologies  thatmay  become 
prevalent  in  the  future.  Another  benefit  is  an  improved  understanding  of  the  specific  components  of  VR  technology 
that  would  be  most  helpful  in  information  visualization.  Thus,  future  development  can  focus  on  using  specific 
mechanisms  of  VEs  to  improve  analysis.  The  final  result  would  be  better  visualizations  for  scientific  inquiry. 

SIS  we  »k  to  design  our  interfere  was  to  use  principles  from  human-computer  interaction  end  cognitive 
science3  Our  interface  was  shown  to  facilitate  a  scientist’s  ability  to  perform  data  analysis  using  scientific 
visualizations  This  finding  supports  the  idea  that  the  use  of  cognitive  theories  as  design  principles  is  a  viable  and 
uSl  ™ch  to  the  design  of  interfaces.  Further,  our  task  analysis  produced  a  knowledge  representation  diagram 
and  a  process  diagram  that  are  still  being  used  to  guide  future  work  on  scientific r  visualizations ^^for  oimanographic 
data  analysis.  That  is,  these  documents  will  enhance  interface  designs  beyond  the  scope  of  this  project. 

enhance  comprehension  of  remotely  se„*d  oceanogmphic  data  is 
an  important  finding  for  the  Navy.  If  immersive  displays  do  not  provide  further  information  to  scientists,  then 
can  be  developed  on  less  expensive,  nonimmersive  systems.  This  also  implies  that  future  work  in 
Te  development  of  visualixiions  can  focus  on  optimizing  the  presentation  of  mfoim.tlon  m  nonimmersive 

formats. 

Alternatively,  future  research  can  investigate  the  applications  for  which  immersive  displays  would  be  more  useful. 
One  possibility  is  that  immersive  displays  would  be  better  suited  to  scientific  visualizations  in  whic  an  ana  ys 
had  to  view  a  changing  environment.  In  our  studies,  the  analyst  could  make  inferences  about  the  ocean  based  on 
single  time-steps.  When  the  task  requires  the  viewing  of  multiple  time-steps,  an  advantage  for  immersion  may 

appear. 


Svatial  Cognition  in  Virtual  Environments  ^  ,  .  , 

Kidual  differences  in  spatial  representations  used  to  navigate  in  virtual  environments  must  be  taken  into  account 
when  designing  any  system  that  requires  navigation.  This  is  particularly  true  when  the  designer  needs  to  make 
tools  for  navigation  available  to  the  users.  Future  research  needs  to  investigate  how  designers  can  fully  account  for 
individual  differences  in  spatial  cognition  in  the  design  of  virtual  interfaces.  It  is  likely  that  different  sets  of  tools 
will  better  facilitate  effective  navigation  for  users  with  each  type  of  spatial  representation. 

Another  issue  addressed  by  this  research  is  the  use  of  VEs  in  training.  As  VE  technology  has  increased,  there  has 
been  a  push  to  use  virtual  worlds  as  training  environments.  The  benefits  of  using  VEs  m  this  way  seem  numerous 
(e.g.  Loomis  et  al.,  1999).  For  example,  training  a  firefighter  in  a  virtual  building  is  much  safer  than  training  one 
in  a  real  burning  building.  These  potential  benefits  will  not  mean  much,  however,  if  the  training  in  a  VE  does  not 
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transfer  to  the  real  world.  Our  findings  indicate  that  many  subjects  used  strategies  in  a  VE  to  enhance  their  ability 
to  nerform  object  localization.  More  research  on  how  these  strategies  are  applied  in  the  real  world  is  needed  to 
enhance™  e  design  of  VE  navigation  tools.  Alternatively,  reliance  on  VE  navigation  tools  that  are  not  available  in 
the  real  world  may  lead  to  performance  during  training  that  is  unachievable  m  the  real  world.  These  possibilities 
must  beludied  more  careMly  and  weighed  into  the  design  of  VEs  used  for  training  m  any  real-world  task. 
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